Cue utilization and strategy application in stable and unstable dynamic environments

http://dx.doi.org/10.1016/j.cogsys.2010.12.004 Copyright © 2011, Elsevie

Learning in a changing environment

Multiple cue probability learning studies have typically focused on stationary environments. We present three experiments investigating learning in changing environments. A fine-grained analysis of the learning dynamics shows that participants were responsive to both abrupt and gradual changes in cue-outcome relations. We found no evidence that participants adapted to these types of change in qualitatively different ways. Also, in contrast to earlier claims that these tasks are learned implicitly, participants showed good insight into what they learned. By fitting formal learning models, we investigated whether participants learned global functional relationships or made localized predictions from similar experienced exemplars. Both a local (the Associative Learning Model) and a global learning model (the novel Bayesian Linear Filter) fitted the data of the first two experiments. However, the results of Experiment 3, which was specifically designed to discriminate between local and global learning models, provided more support for global learning models. Finally, we present a novel model to account for the cue competition effects found in previous research and displayed by some of our participants

Identifiability of Gaussian Bayesian bandit models

The Kalman filter, combined with heuristic choice rules such as softmax, UCB, and Thompson sampling, has been a popular model to identify the role of uncertainty in exploration in human reinforcement learning. Here we show that the Kalman filter combined with a softmax or UCB choice rule is not fully identifiable. By this structural identifiability, we mean that with unlimited data, the true parameter values are determinable. Perhaps surprisingly, the Kalman filter with Thompson sampling is fully identifiable

Models of probabilistic category learning in Parkinson's disease: Strategy use and the effects of L-dopa

Probabilistic category learning (PCL) has become an increasingly popular paradigm to study the brain bases of learning and memory. It has been argued that PCL relies on procedural habit learning, which is impaired in Parkinson's disease (PD). However, as PD patients were typically tested under medication, it is possible that levodopa (L-dopa) caused impaired performance in PCL. We present formal models of rule-based strategy switching in PCL, to re-analyse the data from [Jahanshahi, M., Wilkinson, L, Gahir, H., Dharminda, A., & Lagnado, D.A., (2009). Medication impairs probabilistic classification learning in Parkinson's disease. Manuscript submitted for publication] comparing PD patients on and off medication (within subjects) to matched controls. Our analysis shows that PD patients followed a similar strategy switch process as controls when off medication, but not when on medication. On medication, PD patients mainly followed a random guessing strategy, with only few switching to the better Single Cue strategies. PD patients on medication and controls made more use of the optimal Multi-Cue strategy. In addition, while controls and PD patients off medication only switched to strategies which did not decrease performance, strategy switches of PD patients on medication were not always directed as such. Finally, results indicated that PD patients on medication responded according to a probability matching strategy indicative of associative learning, while the behaviour of PD patients off medication and controls was consistent with a rule-based hypothesis testing procedure. (C) 2009 Elsevier Inc. All rights reserved

Semantic cross-scale numerical anchoring

Anchoring effects are robust, varied and can be consequential. Researchers have provided a variety of alternative explanations for these effects. More recently, it has become apparent that anchoring effects might be produced by a variety of different processes, either acting simultaneously, or else individually in distinct situations. An unresolved issue is whether anchoring, aside from simple numeric priming, can transcend scales. That is, is it necessary that the anchor value and the target judgment are expressed in the same units? Despite some theoretical predictions to the contrary, this paper demonstrates semantic cross-scale anchoring in four experiments. Such effects are important for the direction of future theorising on the causes of anchoring effects and understanding the scope of their consequences in applied domains

Social Influence on Risk Perception During Adolescence.

Adolescence is a period of life in which peer relationships become increasingly important. Adolescents have a greater likelihood of taking risks when they are with peers rather than alone. In this study, we investigated the development of social influence on risk perception from late childhood through adulthood. Five hundred and sixty-three participants rated the riskiness of everyday situations and were then informed about the ratings of a social-influence group (teenagers or adults) before rating each situation again. All age groups showed a significant social-influence effect, changing their risk ratings in the direction of the provided ratings; this social-influence effect decreased with age. Most age groups adjusted their ratings more to conform to the ratings of the adult social-influence group than to the ratings of the teenager social-influence group. Only young adolescents were more strongly influenced by the teenager social-influence group than they were by the adult social-influence group, which suggests that to early adolescents, the opinions of other teenagers about risk matter more than the opinions of adults

Transfer of Learned Opponent Models in Zero Sum Games

Human learning transfer takes advantage of important cognitive building blocks such as an abstract representation of concepts underlying tasks and causal models of the environment. One way to build abstract representations of the environment when the task involves interactions with others is to build a model of the opponent that may inform what actions they are likely to take next. In this study, we explore opponent modelling and its role in learning transfer by letting human participants play different games against the same computer agent, who possesses human-like theory of mind abilities with a limited degree of iterated reasoning. We find that participants deviate from Nash equilibrium play and learn to adapt to the opponent’s strategy to exploit it. Moreover, we show that participants transfer their learning to new games and that this transfer is moderated by the level of sophistication of the opponent. Computational modelling shows that it is likely that players start each game using a model-based learning strategy that facilitates generalisation and opponent model transfer, but then switch to behaviour that is consistent with a model-free learning strategy in the later stages of the interaction

Transfer of learned opponent models in repeated games

Human learning transfer takes advantage of important cognitive building blocks such as an abstract representation of concepts underlying tasks and causal models of the environment. One way to build abstract representations of the environment when the task involves interactions with others is to build a model of the opponent that may inform what actions they are likely to take next. In this study, we explore opponent modelling and its role in learning transfer by letting human participants play different games against the same computer agent, who possesses human-like theory of mind abilities with a limited degree of iterated reasoning. We find that participants deviate from Nash equilibrium play and learn to adapt to the opponent's strategy to exploit it. Moreover, we show that participants transfer their learning to new games and that this transfer is moderated by the level of sophistication of the opponent. Computational modelling shows that it is likely that players start each game using a model-based learning strategy that facilitates generalisation and opponent model transfer, but then switch to behaviour that is consistent with a model-free learning strategy in the later stages of the interaction